*»«r»>. 


■’ > 
’f> 


ffe 

■:4 

M 


/f[)  7V7 OOO 


TECHNICAL  REPORT  NO.  55 


RELIABILITY  GROWTH  MODELING 


Larry  H.  Crow 
August  1972 

Reproduced  From 
Best  Available  Copy 


Approved  for  public  release; 
distribution  unlimited. 


AMCMS  Code  2270.1031 


U.S.  ARMY  MATERIEL  SYSTEMS  ANALYSIS  AGENCY 
ABERDEEN  PROVING  GROUND,  MARYLAND 


UNCLASSIFIED 

_ Security  On 


i  DOCUMENT  CONTROL  DATA  -RAD 

1  ORKlNATlNE  ACTIVITY  fC?A4p—»d# 

U.S.  Army  Material  Systems  Analysis  Agency 

Aberdeen  Proving  Ground,  Maryland 

141.  REPORT  SECURITY  Cl.  A  SSI  FlC  ATION 

UNCLASSIFIED 

a*.  •«oup 

1.  REPORT  TITL* 

RELIABILITY  GROWTH  MODELING 

4.  DIIC4IFTIVK  MCTII  (Trt*  «<  «|i«K  Inelumlr .  4aft) 

T  author!*!  (Fin)  mm*,  mUdtm  Initial,  laal  nmm»)  ~ 


Larry  H.  Crow 


1.  REPORT  OPT* 

7«.  TOTAL.  NO-  OP  PAC*S 

.55 

7ft.  NO-  OP  REPS 

43 

1 M.  CONTRACT  OR  ORANT  N«. 

project  no.  AMCMS  Code  2270.1031 

c. 

i. 

M.  ORIOlNATOR'l  REPORT  NUMIERID 

Technical  Report  No.  55 

•b.  OTHER  REPORT  NOUI  (Any  othmr  number*  (Tl«f  m»y  bm  *»•<#>• 
thl*  rmpoM) 

Approved  for  public  release;  distribution  unlimited. 

II.  lUPPLlMINTART  NOTE! 

11.  aRQNSORINO  military  activity 

U.S.  Army  Materiel  Command 

Washington,  D.C. 

111.  All  TRAC  T 

Mathematical  models  describing  the  changes  in  system  reliability  during  a 
development  program  are  called  "reliability  growth"  models.  The  invent  of  this 
report  is  to  familiarize  the  reader  with  reliability  growth  modeling  and  to  discuss 
its  usage  as  a  tool  for  program  managers.  The  discussion  is  supplemented  by 
numerical  examples  illustrating  several  models.  Also,  a  review  of  a  number  of 
reliability  growth  models  currently  available  and  a  bibliography  on  reliability 
growth  are  given. 


UNCLASSIFIED 


Security  Clfiilttcition 


U.S.  ARMY  MATIKILL  SYSTLMS  ANALYSIS  ACiliNCY 


TLCliN  I  CAL  RLl’ORT  NO.  $S 


LllCrow/dga 

Aberdeen  I’roving  Ground,  Md. 
August  1972 


RLUAB1LUY  GROWTH  MOULTING 


ABSTRACT 

Mathematical  models  describing  the  changes  in  system  reliability 
during  a  development  program  are  called  "reliability  growth"  models. 

The  intent  of  this  report  is  to  familiarize  the  reader  with  reliability 
growth  modeling  and  to  discuss  its  usage  as  a  tool  for  program  managers. 
The  discussion  is  supplemented  by  numerical  examples  illustrating 
several  models.  Also,  a  review  or  a  number  of  reliability  growth  models 
currently  available  and  a  bibliography  on  reliability  growth  are  given. 
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The  govcrnm«.nt 's  materiel  acqui  s  i  t  i  on  prui  i’ss  l'or  new  systems 
requiring  development  is  invariably  complex  and  «J  1  ffi  vn  1 1  for  many 
reasons .  benera ) I y  the  materiel  systems  involve  appreciable  departures 
f  rom  existing  systems  eoinii.on  to  the  eoiiimere  i a  1  world.  The  development 
of  tile  materiel  systems  frequently  involve  new  technology  and  a  eiia  1  i  e:i.;e 
to  the  state  of  the  art.  The  pressure  to  improve  rombat  effectiveness 
invariably  requires  compressed  time  scales  .is  . empaied  to  similar 
developments  in  the  eonmieveial  world.  The  requirements  for  reliability 
and  performance  parameters  are  usually  highly  demanding  and  the  combat 
and  climatic  environments  invariably  cover  broad  spec  trams.  Procurement 
actions,  of  necessity,  must  adhere  to  uncommon  constraints,  c .  g .  lowest 
bid.  The  exercises  in  controls  are  also  somewhat  constrained.  The 
procedures  for  management  ot  resources,  funding,  manpower,  contracting 
and  time  scales  present  formidable  management  tasks.  Vendor- vendee 
relationships  are  somewhat  different  from  those  of  the  commercial 
world.  !hc  requirements,  thei i  development,  coord i nut i oil  and  communi¬ 
cation  present  dynamic  and  difficult  problems. 

It  is,  therefore,  essential  that  we  continual 1>  strive  for  improve¬ 
ment  in  tile  management  ot  the  materiel  acquis  it  ion  process.  One  avenue 
for  improvement  which  is  currently  in  a  formative  or  emergent  stage  is 
that  of  re  1  lability  grow  til  mode  I  tag.  Reliability  growth  modeling,  has 
tile  potential  to  accompli:. it  several  things: 

a.  To  place  into  prospective  the  relationship  of  requirements 
with  respect  to  stage  of  materiel  acquisition. 

b.  To  focus  attention  on  the  need  for  quantitatively  tracking 
reliability  throughout  the  materiel  acquisition  stages. 

e.  To  focus  attention  on  failures  to  meet  goals  m  order  that 
corrective  action  may  tie  taken  promptly. 

d.  To  aid  in  the  allocation  and  reallocation  of  resources  to 
achieve  goals  on  schedule  and  within  constraints. 
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«,  To  establish  a  logical  basis  for  projecting  reliability  growth 
so  that  this  consideration  may  properly  be  included  in  the  decision 
making  process. 

Most  often  there  is  not  a  meeting  of  the  minds  as  to  the  stage  of 
development  where  the  requirements  stated  in  Materiel  Needs  (MN's) 
documents  are  to  be  met.  It  is  believed  that  most  of  the  requirements 
are  generated  with  the  thought  in  mind  that  they  represent  what  the 
user  desires  of  the  materiel  at  the  time  it  is  fielded  or  during  field 
use.  Current  practice  seems  to  overlook  this  and  assumes  that  the 
requirement  is  to  be  mot  at  all  stages  of  the  materiel  acquisition  and 
test  process.  Frequently,  decisions  at  all  levels  are  affected  while 
these  anomalies  exists  It  is  therefore  quite  important  that  management 
tools  be  designed  to  recognize  the  transition,  in  reliability  growth 

I 

throughout  the  materiel  acquisition  and  test  process  and  that  interim 
goals  be  established  accordingly.  This  should  be  of  considerable 
assistance  in  achieving  a  sounder  management  and  decision  process 
during  the  materiel  acquisition  life  cycle.  The  other  aforementioned 
points  are,  also,  of  great  importance  because  the  failures  to  quanti¬ 
tatively  track  materiel  reliability  most  often  leads  to  appalling 
surprises,  consternation  and  costly,  untimely  decisions.  Management's 
degrees  of  freedom,  quality  of  decision,  and  alternatives  are  highly 
dependent  on  quantitative  measures  of  where  we  are  and  where  are  we 
going  as  often  as  practicable  throughout  the  life  cycle.  Corrective 
actions  must  be  based  on  adequate  knowledge  of  the  reliability  and 
performance  of  the  materiel  systems  during  milestones  of  the  materiel 
acquisition  process  and  the  timely  reorientation  of  resources  can  be 
an  important  ingredient  in  effective  management. 
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1.  INTRODUCTION  AND  SUMMARY 


A  topic  of  considerable  interest  in  current  reliability  studies  is 
that  of  accounting  for  the  changes  in  system  reliability  that  result 
from  design  and  engineering  modifications  during  a  development  program. 

A  development  program  is  generally  recognized  as  being  a  necessity  for 
most  systems  since  they  usually  exhibit  initial  design  and  engineering 
deficiencies.  Attempts  are  made  during  the  development  program  to 
find  and  remove  these  deficiencies  to  a  point  vhore  certain  levels  of 
performance  with  respect  to  reliability  and  other  requirements  are  met. 

It  is  usually  assumed  that  the  system  reliability  will  increase 
during  the  development  program  and,  thus,  mathematical  models  describ¬ 
ing  this  phenomenon  have  come  to  be  called  "reliability  growth"  models. 
The  purpose  of  this  report  is  to  acquaint  tho  reader  with  the  benefits 
of  reliability  growth  modeling  as  a  tool  for  program  managers  and  to 
familiarize  him  with  some  of  those  models  and  their  applications. 

Specifically,  in  Section  2  reliability  growth  modeling  is  discussed 
as  a  management  tool  for  program  managers.  Some  background  on  the 
general  area  of  reliability  growth  modeling  is  given  in  Section  3  and 
in  Section  4  several  models  are  illustrated  by  numerical  examples. 

Section  5  discusses  briefly  certain  practical  considerations  which 
should  be  taken  into  account,  when  choosing  a  reliability  growth  model. 
Appendix  A  presents  a  review  of  a  number  of  reliability  growth  models 
currently  available  and  Appendix  B  is  a  bibliography  on  reliability 
growth. 
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2.  RELIABILITY  GROWTH  MODELING  -  A  MANAGEMENT  TOOL 


Although  the  purpose  of  a  development  program  for  a  system  is  to 
increase  the  system’s  reliability  to  a  level  acceptable  to  the  user, 
it  is  unfortunately  true  that  many  programs  fail  to  achieve  this  goal 
within  initial  cost,  manpower  and  time  limitations.  The  major  cause 
for  these  failures  is  associated  with  the  complexities  of  many  of 
today's  systems  which  make  it  extremely  difficult  to  determine  before 
development  the  relationships  between  the  requirod  final  reliability, 
the  system's  basic  design  and  tho  total  development  effort.  As  a  result, 
the  program  managers  may  bo  facod  with  the  responsibility  of  planning  a 
development  program  without  firm  rationale  as  to  how  to  allocate  the 
various  resources  and  how  to  dotormino  tho  project  milestones. 

The  purpose  of  the  project  milestones  is  to  set  goals  which  will 
gv.ido  the  development  program  stop  by  stop  so  that  the  reliability 
requirement  will  bo  mot  at  least  within  tho  final  stage  of  development. 
However,  if  tho  necessary  development  effort  is  not  known  beforehand 
then  tho  program  managers  must  try  to  dotormino  whether  or  not  the 
milestones  are  being  mot  as  the  development  program  progresses.  If, 
for  example,  the  reliability  ef  the  system  at  tho  present  milestone  is 
found  to  be  below  a  specified  levol  then  additional  development  effort 
may  be  necessary  to  moot  the  required  level  at  tho  next  milestone. 
Furthermore,  given  tho  rosults  of  the  development  program  up  to  the 
present  time  the  program  managers  may  quostion  tho  realism  of  meeting 
the  reliability  requirements  of  futuro  milestones  with  present 
resources. 

Therefore,  in  light  of  various  uncertainties  the  program  managers 
must  try  to  ostimate  thu  progress  of  the  development  program  up  to  the 
present  time  and  relate  this  by  some  means  to  the  future  development  of 
the  system.  In  general,  however,  it  is  often  a  difficult  task  for  one 
to  obtain  good  estimates  of  tho  progress  of  the  development  program  and 
project  futuro  progress.  For  oxamplo,  to  directly  estimate  the  system’s 
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reliability  at  some  timo  during  development,  a  certain  number  of  systems, 
which  are  homogeneous  with  respect  to  tho  design  configuration  at  that 
time,  must  be  tested.  The  number  of  systems  tested  would  depend  on 
the  precision  desired  in  the  estimate.  In  many  cases  this  approach  is 
extremely  costly  and  may  even  be  impossible  if  the  needed  prototypes 
cannot  be  produced.  Also,  it  should  be  remarked  that  the  proportion 
of  successful  tests  of  the  system  up  to  the  present  time  measures  only 
the  average  reliability  of  past  tests  and  Joes  not  measure  the  present 
reliability  or  aid  in  predicting  tho  future  reliability  of  the  system. 

The  average  reliability  will  lio  botween  the  initial  and  final  reli¬ 
ability  of  the  system  in  most  casos. 

If  tho  progress  of  the  development  program  cannot  bo  reasonably 
estimated  then  it  is  clear  that  tho  program  managors  may  havo  difficulty 
planning  and  conducting  tho  program  to  insure  that  the  roquired 
reliability  will  be  mot  as  scheduled.  It  is  apparent,  too,  that  program 
managers  generally  nood  specialized  techniques  and  methodology  in  this 
regard.  The  area  of  reliability  growth  modeling  is  a  management  tool 
directed  toward  this  need  of  tho  program  managors. 


li 


3.  BACKGROUND 


The  development  of  a  system  generally  evolves  as  a  repeated  process 
of  system  examination  and  testing,  determination  of  system  failure 
modes,  and  design  and  engineering  changes  as  attempts  to  eliminate 
these  modes.  The  problem  of  accounting  for  the  resulting  changes  in 
the  system's  reliability  during  a  development  program  by  mathematical 
models  has  been  of  interest  for  a  number  of  years.  Much  of  the 
research  on  this  subject,  however,  has  not  been  published  in  the  open 
literature  and  is  often  difficult  to  obtain. 

The  central  purpose  of  most  reliability  growth  models  includes 
one  or  both  of  the  following  objectives: 

•  Inference  on  the  present  system  reliability; 

•  Projection  on  rhe  system  reliability  at  some  future 
development  time. 

Most  of  the  reliability  growth  models  considered  in  the  literature 
assume  that  a  mathematical  formula  (or  curve),  as  a  function  of  time, 
represents  the  reliability  of  the  system  during  the  development 
program.  It  is  commonly  assumed,  also,  that  these  curves  are  non¬ 
decreasing.  That  is,  once  the  system's  reliability  has  reached  a 
certain  level,  it  will  not  drop  below  this  level  during  the  remainder 
of  the  development  program.  It  is  important  to  note  that  this  is 
equivalent  to  assumir*,  that  any  design  or  engineering  changes  made 
during  the  development  program  do  not  decrease  the  system's  reliability. 

If,  before  the  development  program  has  begun,  the  exact  shape  of 
the  reliability  growth  curve  is  known  for  a  certain  combination  of 
system  design  and  development  effort,  then  the  model  is  a  deterministic 
one.  In  this  case  the  amount  of  development  effort  needed  to  meet  the 
reliability  requirement  could  be  determined,  and  the  sufficiency  of 
the  design  would,  also,  be  known. 

In  most  situations  encountered  in  practice,  the  exact  shape  of 
the  reliability  growth  curve  will  not  be  known  before  the  development 
program  begins.  The  p-ogram  manager  may,  however,  be  willing  to 
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assume  that  the  curve  belongs  to  some  particular  class  of  parametric 
reliability  growth  curves.  This  is  analogous  to  life  testing  situations 
when  the  experimenter  assumes  that  the  life  distribution  of  the  items 
is  a  member  of  some  parametric  class  such  as  the  exponential,  gamma,  or 
Weibull  families.  The  analysis  then  reduces  to  a  statistical  problem  of 
estimating  the  unknown  parameters  from  the  experimental  data.  These 
estimates  may  be  revised  as  more  data  are  obtained  as  the  development 
program  progresses.  Using  these  estimates,  the  program  manager  can 

monitor  and  project  the  reliability  of  the  system  and  make  necessary 

\ 

decisions  accordingly. 

v 

Some  Bayesian  reliability  growth  models  1  ave,  also,  appeared  in 
the  literature.  This  approach  assumes  that  th>  unknown  parameters  of 
the  growth  curve  are  themselves  random  variables  governed  by  appropriate 
prior  probability  distributions.  Generally,  the  form  of  the  prior 
probability  distributions  are  assumed  to  be  known,  and  the  unknown 
parameters  of  the  relir  lity  growth  curve  may  be  estimated  w’l'tjh  the 
aid  of  Bayes  Theorem. 

Other  models  considered  in  the  literature  may  be  classified  as 
nonparametric.  This  approach  allows  for  the  estimation  of  the  present 
system  reliability  from  experimental  data  without  attempting  to  fit  a 
particular  parametric  curve.  The  estimates  are  usually  conservative 
and  projections  on  future  system  reliability  are  generally  not  possible. 

A  review  of  some  reliability  growth  models  which  have  appeared  in 
the  literature  are  given  in  Appendix  A  and  a  bibliography  on  reliability 
growth  is  given  in  Appendix  B. 


4.  EXAMPLES 


To  acquaint  the  reader  with  the  practical  applications  of  reliability 
growth  modeling,  several  numerical  examples  shall  be  presented  in  this 
section.  The  models  illustrated  by  these  examples  are,  also,  discussed 
in  Appendix  A  and  appropriate  reference  to  this  Appendix  will  be  given 
for  each  model  considered. 

Example  1.  The  Duane  Model  (see  Appendix  A,  Model  7)  will  be  illustrated 
by  this  example.  Briefly,  this  model  assumes  that 

X (T)  =  KT"“, 

where  X (T)  is  the  cumulative  failure  rate  of  the  system  at  operating 
time  T  and  K>  0,  0  £  a  <_  1  are  parameters  to  be  determined  before  the 
start  of  the  development  program.  The  example  is  from  "Reliability 
Planning  and  Management  -  RPM"  by  J.  D.  Selby  and  S.  G.  Miller  (see 
Appendix  B,  Reference  34). 


A  customer  generates  a  requirement  to  design,  develop,  and  deliver 
a  new  avionics  equipment  which  violates  no  laws  of  physics,  is  expected 
to  be  within  the  state  of  the  art,  has  a  lead  time  of  36  months,  and 
an  apportioned  MTBF  from  a  higher  level  system  requirement  of  150  hours. 

The  reliability  program  is  to  be  in  accordance  with  MIL-STD-785  with 
testing  per  MIL-STD-781  test  level  F;  First  Article  Configuration 
Inspection  (FACI)  and  configuration  control  are  required  on  the  first 
production  item. 

Viewing  these  requirements,  a  contractor  first  determines  a 
functional  implementation  plan  and  equipment  schedule  based  on  the 
technical  exhibits,  previous  experience,  equipment  complexity,  and 
program  planning  judgments.  Let  us  assume  the  case  of  a  responsible 
contractor  who  assesses  the  requirement  as  being  within  the  state  of 
the  art  and  represents  an  equipment  cf  UK  parts  complexity.  Using 
MIL-STD-217A  as  a  departure  point,  a  prediction  of  220  hours  is  calculated. 
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made  possible  by  use  of  screened  parts,  applied  under  exacting  application 
and  derating  criteria.  This  prediction  meets  the  first  RPM  criteria  by 
exceeding  the  minimum  125%  of  specified  requirement. 

The  schedule  milestones  are  established,  based  on  past  development 
experience,  resulting  in  15  months  for  design,  6  additional  months  for 
initial  hardware  manufacture  and  ambient  test,  12  months  for  evaluation 
testing,  and  3  months  for  final  change  documentation  and  incorporation 
prior  to  production  FA 'I  and  configuration  control  constraints. 

The  second  and  third  criteria  of  the  model  state  that  for  a  new 
design  the  initial  hardware  MTBF  will  be  10%  of  that  predicted,  and 
reliability  growth  will  follow  the  Duane  postulate.  Employing  these 
criteria,  a  reliability  initial  estimate  and  growth  requirement  based 
on  the  specifics  of  the  selected  implementation  now  can  be  structured. 

The  initial  capability  for  this  new  system  is  thus  dimensioned  as 
22  hours,  10%  of  the  pred  cted  MTBF.  A  growth  rate  of  .5  is  planned 
based  upon  a  comprehensive  reliability  program  executed  through  com¬ 
petent  implementation  of  MIL-STD-785  and  MIL-STD-781.  The  growth 
requirements.  Figure  1,  indicate  that  compliance  can  be  achieved  at 
4800  hours  of  te*t  time. 

In  implementation  planning,  optimization  of  the  reliability  program 
about  individual  program  priorities  is  desirable  and  required.  In  this 
example,  the  contractor  elects  to  consider  optimization  in  three  cases 
of  program  structuring:  minimum  assets,  compliant  time,  and  least  risk. 

The  objective  is  clearly  to  dimension  and  evaluate  the  alternatives  and 
activity  necessary  to  achieve  required  equipment  reliability  during 
development  and  prior  to  the  production  and  user  constraints.  Based  on 
a  review  of  tost  experience  with  complex  systems,  it  has  been  established 
for  GE/AES  products  that  200  hours  of  test  operate  time  can  reasonably 
be  achieved  per  system  month  of  effort. 
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Operating  within  these  bounds,  let  us  now  consider  the  test  plan 
options  open  to  the  contractor  to  develop  a  compliant  product.  The 
first  option,  minimum  assets,  requires  one  system  tested  continuously 
for  24  months  representing  the  least  number  of  systems.  The  second 
option,  a  time  compliant  test,  requires  two  systems  tested  continuously 
and  concurrently  for  12  months.  The  third  option,  least  risk,  requires 
three  systems  tested  continuously  and  concurrently  for  8  months, 
accommodating  additional  time  for  reaction  to  contingency  including 
growth  of  up  to  25%  in  product  complexity.  This  growth  reflects  the 
case  where  an  11K  part  original  estimate  grows  during  detail  implementation 
to  a  14K  part  system. 

From  these  optimizations,  the  magnitude  of  the  program,  kinds  of 
disciplines,  and  available  tradeoffs  bounding  a  successful  program  are 
now  clear  to  management. 
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Example  2.  This  example  and  the  following  two  examples  illustrate  the 
use  of  the  Gompertz  equation  considered  by  Virone  (see  Appendix  A, 

Model  6),  These  examples  are  fromVirene's  paper  entitled,  "Reliability 
Growth  and  Its  Upper  Limit"  (see  Appendix  B,  Reference  38). 

The  Gompertz  equation  is 


R  =  abC  , 

where  a  is  the  (unknown)  upper  limit  approached  by  the  reliability,  R, 
as  time  t  — >  «,  and  0<b<l,0<c<l  are  parameters  to  be  estimated 
from  test  data. 

The  following  steps  are  necessary  for  estimating  a,  b,  c  by  Virene's 
method. 

A.  Arrange  the  currently  available  data,  values  for  t  and  R,  in 
columns, 

B.  Calculate  values  for  log  R. 

C.  Divide  the  column  of  values  for  log  R  into  three  equal-size 
groups,  each  containing  n  items. 

D.  Add  the  values  of  log  R  in  each  group,  obtaining  sums  identified 
as  Sj,  S2,  and  S^,  starting  with  the  lowest  values  of  log  R. 

E.  Calculate  the  value  for  c. 

F.  Calculate  the  value  for  a. 

G.  Calculate  the  value  for  b. 

H.  Set  a  value  for  t,  the  point  to  which  the  projection  is  to  be 
extended,  e.g.,  100,000  hours,  30  months,  etc. 

I.  Substitute  the  values  for  a,  b,  and  c  into  the  Gompertz  equation 
to  obtain  the  value  for  R  at  point  t. 

The  ceiling  value  of  R  is  a.  That  is,  at  any  point  in  rime  the 
currently  available  data  can  be  used  to  calculate  a  value  for  a,  which 
is  the  highest  value  that  R  is  likely  to  reach.  Values  for  a,  b,  and  c 
are  calculated  by  the  following  equations: 
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Whore: 

Sj  *  Summation  of  log  R  in  group  1. 

S2  ■  Summation  of  log  R  in  group  2. 

S^  *  Summation  of  log  R  in  group  3. 

n  *  Number  of  previously  determined  reliability  values  in  a 

group . 


Suppose  that  it  is  contractually  required  that  a  certain  device 
have  an  assessed  reliability  of  0,90  at  the  end  of  a  10-month  design 
and  development  period.  What  will  be  the  assessed  reliability  at  the 
contract  completion  date,  and  the  maximum  achievable  reliability  without 
major  redesign? 

Starting  with  the  available  assessment  data  at  the  end  of  five 
months,  convert  the  reliabilities  to  percentages  and  calculate  the  log 
R  values: 
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Conclusions : 


The  upper  limit  for  achievable  reliability  is  indicated  by  a,  which 
was  found  to  be  92  percent. 

When : 

t  ■  10  months,  as  in  this  contract 
0  729^ 

R  *  92.0  (0.608)u,/  s 
log  R  =  log  92.0  +  0.72910  log  0.608 

=  1.964  +  0.0427  (-0.216) 

*  1.955 

R  =  90.2,  assessed  percent  reliability 
at  tho  end  of  10  months. 

Since  tho  required  reliability  is  90  percent,  the  calculations 
indicate  that  the  requirement  will  be  met,  hut  just  barely.  Unless 
there  is  a  possibility  of  making  a  significant  improvement  by  a  design 
change  within  the  remaining  5-month  period  of  the  contract,  the 
achievable  reliability  should  be  safeguarded  by  such  means  as  the  use 
of  extreme  care  in  material  and  process  controls,  inspection  procedures, 
handling  methods  and  transportation  arrangements. 

Goodness  of  Fit 

Use  of  the  Gompertz  equation  should  depend  on  its  suitability  in 
describing  the  present  data.  To  determine  tho  goodness  of  fit  of  the 
equation 


0  720 

R  =  92.0  (0.608)”'  *" 

calculate  reliability  values  for  t  -  0,  1,  2,  3,  4,  5  and  then  compare 
these  values  with  the  presently  available  reliability  values. 
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If  T 


R  R 


calculated  from 
equation 

data 

avai lable 

0 

56.0 

56.0 

1 

64.1 

64.0 

2 

70.6 

70.5 

3 

75.9 

76.0 

4 

80.0 

80.0 

5 

83.2 

83.0 

Tho  Gomportz  curve  is 

a  good  fit  for  tho  data  used. 

since  the 

equation  reproduces  the  available  data  with  less  than  1%  error. 

Example  3.  Data  from  the  Lunar  Orbiter  is  tabulated  below  in  Table 
1.  What  will  the  reliability  be  in  January  1965,  and  what  will  be 
the  upper  reliability  limit  without  redesign? 

The  data  in  Table  1  were  based  on  successive  design  analysis  only, 
because  no  test  or  use  data  were  available  at  the  time  the  predictions 
were  made.  The  increase  in  predicted  reliability  reflected  the  increase 
in  detailed  knowledge  about  the  design. 


Table  1 


Time 

Reliability 
Converted 
to  H) 

(t) 

m 

log  R 

Group  1 

10  Juno  1964 
( 1  July 

51.0 

59.4 

1.708 

1.774 

Group  2 

f 2  August 
\ 3  September 

63.8 

66.6 

■ 

Group  3 

/4  October 
(5  Novomber 

68.8 

65.5 

1.838 

1.816 

1.708  1.80S  1.838 

1.774  1.823  1.816 

S2  =  3.628  S3  ■=  3.654 


Sj  =  3.482 
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1  1 


»  1.830. 

a  ■  68.1,  the  upper  limit  for  reliability. 

It  was  found  on  the  basis  of  the  first  six  month's  data  and  the 
Gompertz  Model  that  the  reliability  predicted  in  January  1965  would  be 
0.680  (which  it  was)  and  that  the  upper  limit  to  the  predicted  reli¬ 
ability  would  be  0.681,  unless  there  was  a  redesign  of  some  modification 
in  the  evaluation  technique. 

Example  4.  In  the  Aerospace  Program,  the  Launch  Vehicle  History  of 
the  Blue  Scout  indicated  15  successes  in  22  trials.  Determine  the  90% 
and  80%  confidence  limits  of  the  reliability  of  the  Blue  Scout. 

The  data  at  the  end  of  22  launches  are  indicated  in  Table  2. 
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TABLE.  2 


No.  of  Launch 

Success 

Failure 

— r~ 

Pt.  Est. 
Rel.  in  % 

- 5 

u0% 

Confidence 

R 

90 % 

Confidence 

1 

X 

0 

2 

X 

0 

3 

X 

II 

0 

II 

4 

X 

25. 0 

5 

X 

20,0 

6 

X 

16.7 

7 

X 

28.6 

8 

X 

37.  S 

19.9 

14.7 

9 

X 

44.4 

26.8 

21.0 

10 

X 

SO.O 

32.7 

26.7 

11 

X 

S4.5 

37.8 

31.8 

12 

X 

1 

58.2 

42.2 

36.2 

13 

X 

61.7 

46.0 

40.2 

14 

X 

64.2 

49.4  j 

43.7 

15 

X 

66.7 

52.4 

46.8 

16 

X 

68.7 

55.1 

49.6 

17 

X 

65.0 

51.5 

46.3 

18 

X 

66.7 

54.0 

48.8 

19 

X 

63.0 

50.9 

45.9 

20 

X 

65.0 

53.1 

48,2 

21 

X 

66.7 

55.1 

50.3 

22 

X 

68.1 

57.0 

52.3 
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To  determine  the  lower  901  confidence  limit  of  the  reliability  of  the 
Launch  Vehicle,  look  at  the  excerpt  from  a  binomial  reliability  table 
for  15  successes  in  22  trials  and  confidence  level  0.90.  We  find  that 
the  lower  90%  confidence  limit  of  the  Launch  Vehicle  is  0.523.  There¬ 
fore,  we  are  90%  confident  that  the  true  population  reliability  is  at 
least  0.523.  The  lower  80%  confidence  limit  is  determined  in  the  same 
way  and  gives  a  reliability  of  at  least  0,570. 

Based  on  data  from  the  first  15  launches,  we  could  estimate,  using 
the  Gompertz  Model,  the  reliability,  with  say  80%  confidence  after  22 
launches.  (This  would  be  a  projected  reliability  if  data  beyond  15 
launches  were  not  available.)  Using  data  from  Table  2,  Table  3  is 
obtained. 


Table  3 


No.  of 
L 

Launch 

L  ■  X  +  4 

X 

Reliability  in  % 

R 

log  R 

1 

0 

2  / 

0 

3  / 

^  0 

4  "j 

0 

25.0 

1.398 

Group  '' 

5 

1 

20.0 

1.301 

1  , 

6 

2 

16.7 

1.223 

7 

3 

28.6 

1.456 

8 

4 

37.5 

1.574 

Group  ! 

9 

5 

44.4 

1.647 

2 

10 

C 

50.0 

1.699 

11 

7 

54.5 

1.736 

12 

8 

58.2 

1.765 

Group 

13 

9 

61.7 

1.790 

3 

14 

10 

64.2 

1.807 

15 

11 

66.7 

1.824 

16 

12 

68.7 

1.837 

17 

13 

65.0 

1.813 

18 

14 

66.7 

1.824 

19 

15 

63.0 

1.799 

20 

16 

65.0 

1.813 

21 

17 

66.7 

1.824 

22 

18 

68.1 

1.833 
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It  is  assumed  that  there  is  no  reliability  growth  until  the  4th 
launch 


1.398 

1.574 

1.765 

1.301 

1.647 

1.790 

1.223 

1.699 

1.807 

1.456 

1.736 

1.824 

5.378 

S2  ■  6.656 

S3  =  7.186 

0.802 


log  a  = 


1.890 


a  *  77.6,  the  upper  limit  for  reliability. 


(S.  -  S  )  Cl  -  <0 

log  b  =  — ± - 1 - - -  =  -0.740  =  0.260  -  1 

/  <  2 
(1-0 


b  =  0.182 
x 

R  =  abC 


R  = 


77.6(0.182) 


0.802X 


After  22  launches,  L  =  22  and,  since  L  =  X  +  4  in  Table  3,  X  = 
(Reliability  growth  starts  when  L  =  4  or  X  =  0.) 


18. 
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Then: 


R  *  77.6(0. 182)0,802  »  77.6C0.182)0,802 

log  R  »  log  77.6  +  0.80218  log  0.182 
*  1.886 
R  =  76.9% 

After  22  launches,  the  reliability,  based  on  the  Gompertz  equation, 
is  76.9%.  The  actual  value  in  Table  3  is  68,1%. 

Example  5.  This  example  illustrates  a  nonparametric  reliability  growth 
model  introduced  by  Barlow  and  Scheuer  (see  Appendix  A,  Model  5) .  In 
this  model  each  failure  must  be  classified  either  as  inherent  or 
assignable  cause  and  the  development  program  is  conducted  in  K  stages 
with  similar  systems  being  tested  within  each  stage.  Further,  it  is 
assumed  that  the  probability  of  an  inherent  failure,  qQ,  remains 
unchanged  throughout  the  development  program  and  the  probability,  q^, 
of  an  assignable  cause  failure  in  the  i-th  stage  does  not  increase  from 
stage  to  stage.  The  example  is  from  "Reliability  Growth  During  a 
Development  Testing  Program"  by  Richard  F..  Barlow  and  Ernest  M.  Scheuer 
(see  Appendix  B,  Reference  3) . 


Suppose  that  a  development  testing  program  yielded  the  results 
shown  in  Table  4,  Each  stage  of  sampling,  except  the  last,  was 
terminated  when  an  assignable  cause  failure  occurred.  A  redesign  effort 
was  undertaken  to  eliminate  the  cause  of  failure,  so  that  the  test  units 
in  the  succeeding  stage  were  different  from  the  earlier  units  but 
homogeneous  in  any  given  stage.  It  is  remarked  that  this  is  the  defining 
property  of  a  stage,  namely  the  homogeneity  of  all  test  units  therein. 
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Table  4 


Stage 

i 

Inherent 

Failures 

8i 

Assignable  1 
Cause  1 

Failures 
b. 

l 

Successes 

c. 

i 

bi 

isa 

1 

0 

1 

0 

1 

i 

2 

0 

1 

0 

1 

KM 

3 

0 

1 

0 

1 

4 

1 

1 

1 

3 

Rh 

5 

0 

1 

4 

5 

In 

6 

0 

1 

0 

1 

IK 

7 

0 

1 

0 

1 

i 

8 

0 

1 

3 

4 

1/4 

9 

9 

1 

27 

37 

1/28 

TOTALS 

10 

9 

35 

54 

- 

The  maximum  likelihood  estimate  of  an  inherent  failure  q  is 


K  K 
£  a  /  £  (a 
i*l  i=l  1 


bi  *  ci^ 


and  the  maximum  likelihood  estimate  of  q^,  an  assignable  cause  failure 
in  the  i-th  stage,  is 


qi  =  0-qo)  bi/(bi  +  c.) 


i=l,...,K.  The  q. 's  are  the  maximum  likelihood  estimates  of  the  q. 's 
in  general.  Let  denote  the  maximum  likelihood  estimates 
of  q^,q2»«..*qK  subject  to  the  condition  that 


If 


qi  i  q2  -  •**  -  V 


ql  lq2  -  —  -V 
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then 


i*l,...,K.  If 

A  * 

qj  "  *i*l 

for  some  j  (j»l, . . . ,K-1) ,  then  combine  the  observations  in  the  j-th 
and  (j+l)-st  stages  and  compute  the  maximum  likelihood  estimates  of 
the  q^'s  using  again  the  definition  of  given  above  for  the  K-l 
stages  thus  formed.  This  procedure  is  continued  until  the  estimates 
of  the  q^s  form  a  non-increasing  sequence.  These  estimates  are  the 
maximum  likelihood  estimates  of  the  q^'s  subject  to 

ql  -  q2  -  ***  -  V 

For  this  example  it  is  found  that 

q  =  .1852 

“  ^2  ^  q3  “  *3l48, 
q4  =  .4074, 

q5  =  q6  =  q7  "  -34q?. 
qfi  =  .2037,  and 

q^  =  .0291. 
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r 


Furthor,  tho  maximum  likelihood  estimate  for  r  ,  the  reliability  of 
the  system  in  its  final  test  configuration,  is 

r9  -  l-qQ  -  q9  ■  .7857. 

If  no  assumption  of  reliability  growth  were  made  -  that  is,  if  all  test 
units  were  (incorrectly)  supposed  to  be  homogeneous  and  if  no  distinction 
were  made  between  inherent  and  assignable  cause  failures  -  the  estimate 
of  reliability  would  be 

9  9 

r  =  Z  c.  /  E  (a.  +  b.  ♦  c.)  =  .6481. 
s  i*l  i*=l 

To  find  a  conservative  100  (1-a)  percent  lower  confidence  bound 
for  r<j,  it  is  not  necessary  to  distinguish  between  inherent  and 
assignable  cause  failures.  The  procedure  is  to  treat  the  data  as 
though  they  were  homogeneous,  that  is,  as  if  no  reliability  growth 
were  taking  place,  and  use  the  standard  technique  to  obtain  a  one-sided 
lower  confidence  limit  on  a  binomial  parameter  having  observed  s 
successes  in  n  trials.  Using  the  data  in  Table  4  it  is  found  that  a 
lower  95  percent  confidence  bound  on  r^  is  .53.  This  is  a  conservative 
bound  and  is  the  same  estimate  that  would  be  obtained  if  no  reliability 
growth  was  assumed. 

Example  6.  Barlow,  Proschan  and  Scheuer  (see  Appendix  A,  Model  9) 
considered  a  reliability  growth  model  similar  to  the  one  illustrated 
in  tho  previous  example.  The  main  distinction  is  that  in  their  model 
one  need  not  classify  failures  as  inherent  or  assignable  cause. 
Specifically,  their  model  assumes  that  a  system  is  being  modified 
during  K  stages  of  development.  Lot  p^  be  the  system  reliability  at 
the  i-th  stage.  Barlow,  et  al,  obtained  the  maximum  likelihood 
estimates  of  p^jp^, . . • ,p^,  under  the  restriction  that 

Px  1P2  1  •••  iPr 
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Also,  a  conservative  lower  confidence  bound  on  was  obtained  assuming 

pK  >.  max  p.  . 
i<K  1 

Data  consist  of  x^  successes  from  n^  observations  in  stage 
i j i*l , . •  •  i  K. 

To  obtain  the  maximum  likelihood  estimates  of  p1,.,,,p^  subject 
to  the  restriction  that  Pj  i  P2  PK»  first  form  the  ratios 

Xl/nl’X2/,n2**‘*  *XK^nK*  If  xl^nl  —  X2'/n2  —  ***  - .  xK/nK*  then  x^ni 

A 

is  the  MLE  of  p^  If  for  some  j  (j*l , . . . ,K-1) ,  x^/n  >  xj+^/nj+l> 

combine  the  observations  in  the  j-th  and  (j  +  lj-st  stages  and  examine 
the  ratios, 

W*--»Xj-l/nj-l’  (Xj  *  Vl)/(nj  +  nj^* 


Xj+2/nj+2*  ’  *  *  ,XK//nK’ 

for  the  (K  -  1)  stages  thus  formed.  If  these  ratios  are  in  nondecreas- 

A 

ing  order,  they  constitute  the  MLE's  of  Pj,...,p^  with  =  p^+^  = 

(Xj  +  Xj^jj/Cn^.  ♦  if  hot,  continue  the  process  of  combining 

stages  until  the  ratios  are  in  nondecreasing  order.  This  process  need 
be  repeated  at  most  (K  -  1)  times,  and  the  result  is  independent  of 
the  order  in  which  stages  are  combined  to  eliminate  reversals  in  the 
sequence  of  ratios. 

The  example  is  from  "Maximum  Likelihood  Estimation  and  Conservative 
Confidence  Interval  Procedures  in  Reliability  Growth  and  Debugging 
Problems"  by  Barlow,  et  al,  (see  Appendix  B,  Reference  2). 
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The  procedure  is  illustrated  by  the  data  in  the  following  table. 


Stage 

(i) 

Successes 

(Xj) 

No.  of 
Trials 

x./n. 
i  i 

1 

2 

5 

.400 

2 

3 

7 

.429 

3 

3 

8 

.375 

4 

2 

6 

.333 

S 

6 

6 

1.000 

The  process  of  combining  stages  to  get  a  sequence  of  non-decreasing 
ratios  is  summarized  below: 


First 

Second 

Third 

i 

X. 

1 

n. 

i 

x./n. 

l  l 

Combination 

Combination 

Combination 

1 

2 

5 

.400 

.400 

.400 

2 

3 

7 

.429 

I 

1 

3 

4 

3 

2 

8 

6 

.375 

.333 

►  6  *»  ,400 
IT?  .333 

-381 

|i|=  .385 

5 

6 

6 

1.000 

1.000 

1.000 

1.000 

Thus,  we  obtain  the  maximum  likelihood  estimates 


Pj  =  P2  =  p3  =  p4  =  .385,  ps  =  1.000. 

A  conservative  100(l-a)  percent  lower  confidence  bound  on  p^,  the 
reliability  of  the  latest  version  of  the  system,  is  found  by  treating 
the  data  from  the  K  stages  of  the  development  program  as  though  they 
were  homogeneous,  then  applying  the  standard  binomial  approach  to  get  a 
lower  confidence  bound  on  a  binomial  parameter  having  observed 

x  =  x^  successes  in  n  =  ^  n^  trials.  Thus,  to  obtain  the  conserv¬ 
ative  lower  confidence  bound  on  p^,  the  stage-by-stage  history  of  the 
development  program  is  not  needed;  only  the  total  number  of  successes 
and  the  total  number  of  trials. 
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In  this  example,  a  total  of  16  successes  were  observed  in  a  total 
of  32  trials.  A  95  percent  lower  confidence  bound  for  a  single  binomial 
parameter  based  on  these  data  is  found  from  binomial  tables  to  be  .344. 
Thus,  a  conservative  95  percent  lower  confidence  bound  for  p,.  is  .344. 

Example  7.  The  reliability  growth  model  illustrated  by  this  example 
was,  also,  considered  by  Barlow,  Proschan,  and  Scheuer  (see  Appendix  A, 
Model  10).  It  is  assumed,  again,  that  a  system  is  being  modified  during 
K  stages  of  development.  Let  be  the  distribution  of  system  life 
length  at  the  i-th  sttge  of  development.  Because  of  reliability  growth 
the  maximum  likelihood  estimates  of  F^  (t)  ,F,,(t) ,.. .  ,F^(t)  are  obtained 
assuming  that 

F L (t)  <_  F2(t)  <,  ...  £  FK(t) 

for  a  fixed  t  >_  0,  where  F^t)  =  l-F^ft). 

Data  consist  of  independent  life  length  observations 

,i=l,...,K. 

i 

The  maximum  likelihood  estimates  (MLE)  of  F^  (t) ,. . . ,F^(t)  are 

obtained  as  follows.  For  1=1,2,. ,.,K,  obtain  the  empirical 

dist:  bution  function  (t)  from  F^n  (t)  =  m^(t)/n^,  where  nr(t)  = 

l  i  1 

nui;  of  observations  among  Xjki»Xi2’’**,Xin  not  exceec*ing  v* 

If  F.  (t)  >  (t)  >  ...  >  F..  (t),  then  these  constitute  MLE's  of 

lnl  “K 

F (t),F»(t),«..,Fl.(t)  respectively.  Suppose,  on  the  contrary,  the 
reversal , 


m. (t)  m.  .  (t) 

r  l  +  l 

<  - 


n. 

i 


n.  , 
i  +  I 


i 

f 

f 
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occurs.  Then  the  MLE  is  obtained  by  assuming  a  common  value  for 
F^(t)  and  Ct) .  Under  this  assumption  the  MLE  of  this  common  value 
is  obtained  by  pooling  the  observations  from  the  two  distributions  to 
obtain 


mi(t)  +  m1+1(t) 


ni  +  Vl 


as  the  MLE  of  the  common  value  F^(t)  =  Then  examine 


“lW . ViW,  *i|t)  *  Vi(t).  ,i|t’*V!W, 


ni-l 


n.  +  n.  , 
1  1+1 


n.  +  n.  , 
i  i+l 


n 


i+2 


If  these  are  in  decreasing  order,  they  constitute  MLE's  of  Fj(t),.,., 
Fi_i(t),FiCt),Fi+i(t),Fi+2(t),...,FK(t).  If  on  the  other  hand,  a 

reversal  exists,  pool  as  before  to  eliminate  the  reversal  (adding  the 
various  hk  (t)  involved  in  the  reversal  to  obtain  a  new  numerator  and 
adding  the  corresponding  n^  to  obtain  a  new  denominator) .  After  each 
reversal  has  been  eliminated,  we  test  the  resulting  sequence  of  ratios 
to  see  if  they  are  in  decreasing  order.  When  finally  a  sequence  of 

A  A 

decreasing  ratios  is  obtained,  those  constitute  the  MLE's  F^Ct),  F^Ct), 
...,FK(t). 

The  example  is  from  "Maximum  Likelihood  Estimation  and  Conservative 
Confidence  Interval  Procedures  in  Reliability  Growth  and  Debugging 
Problems"  by  Barlow,  et  al,  (see  Appendix  B,  Reference  2). 
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A  development  program  has  two  stages  of  development  with  four 
observations  In  Stage  1  and  six  observations  in  Stage  2.  The  data  are 
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Note,  again,  that  the  procedure  illustrated  above  yields  the  MLE 
for  any  ono,  fixed,  predetermined  value  of  t,  not  for  the  entire 
distribution  function. 


5.  COMMENTS  ON  CHOOSING  A  MODEL 


So  far  the  need  for  reliability  growth  modeling  has  been  discussed 
and  some  background  and  examples  on  this  subject  have  been  given.  The 
next  area  of  interest  is  naturally  the  practical  aspects  of  choosing 
an  appropriate  model.  Because  of  the  lack  of  published  research  in 
this  area,  the  present  section  will  necessarily  address  the  problem 
in  generalities. 

As  in  any  mathematical  model,  reliability  growth  models  are 
idealizations  and  are  based  on  a  number  of  assumptions  which  vary  with 
the  models.  With  parametric  and  Bayesian  models  it  may  be  very 
difficult  or,  in  fact,  impossible  to  verify  the  underlying  assumptions 
in  practice.  Therefore,  a  problem  of  prime  importance  is  the  robustness 
of  the  statistical  procedures  for  the  parametric  and  Bayesian  models; 
i.e.  how  well  will  these  procedures  represent  the  actual  growth  process 
when  the  underlying  assumptions  are  not  entirely  satisfied.  Unfortun¬ 
ately,  sufficient  research  to  answer  this  question  has  not  been  performed. 
In  light  of  this  lack  of  research,  one  should  be  careful  when  choosing 
a  parametric  or  Bayesian  model  since  any  results  obtained  from  the 
model  may  be  crucial  and  expensive  to  both  the  user  and  developer  if 
it  does  not  reasonably  represent  the  growth  process. 

Another  related  factor  when  choosing  a  parametric  reliability 
growth  model  is  whether  statistical  procedures  are  available  for 
estimating  and/or  determining  confidence  statements  on  the  unknown 
parameters.  For  example,  in  the  simple  Lloyd  and  lipow  model  (see 
Appendix  A,  Model  1)  it  is  easily  shown  that  a  "good"  estimator  of  the 
failure  probability  does  not,  in  general,  exist. 

With  little  knowledge  about  the  characteristics  of  the  growth 
process  one  may  wish  to  use  a  nonparametric  reliability  growth  model., 

Of  course,  one  must  pay  a  price  for  this  scarcity  of  knowledge.  The 
major  shortcoming  of  the  nonparametric  models  is  that  lower  confidence 


bounds  tend  to  bo  ovorly  conservative.  The  result  of  this  could  be 
unnecessary  development  cost  since  important,  decisions  concerning  the 
reliability  of  the  system  would  usually  bo  based  on  the  lower  confidence 
bound. 

One  should  keep  in  mind,  too,  that  the  growth  process  is  a  function 
of  tie  development  effort.  If  tho  development  effort  changes  then  one 
■say  wish  to  examine  tho  reliability  growth  model  being  used  to  see  if 
it  is  still  realistic.  If  if  is  possible  that  the  development  effort 
will  chango  in  the  future,  then  one  should  be  cautious  when  projecting 
the  future  reliability  based  on  the  prosent  model  and  past  data. 

Tho  proper  application  of  reliability  growth  models  may  often 
require  a  close,  continuing  relationship  between  the  user,  developer 
and  all  other  interested  parties.  Evoryono  involved  should  realize 
from  the  outset  that  reliability  growth  modeling  is  more  than  a 
statistical  exercise  of  fitting  experimental  data  to  arbitrary 
mathematical  functions.  If  properly  understood  and  used,  reliability 
growth  modoling  can  be  a  very  important  management  tool.  Otherwise, 
it  may  he  useless  and  harmful. 
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APPENDIX  A 


A  REVIEW  OF  SOME  RELIABILITY  GROWTH  MODELS 

This  appendix  describes  a  number  of  reliability  growth  models  which 
are  currently  available.  Each  model  j.s  briefly  described  including  the 
basic  assumptions  that  were  made  in  deriving  the  models.  Technical 
references  are  given  for  each  of  these  models  whore  a  more  complete 
discussion  of  the  model  may  be  found. 

Model  1.  Lloyd  and  Lipow  (see  Appendix  B,  Reference  28)  introduced  a 
reliability  growth  model  for  a  system  which  has  only  one  failure  mode. 
For  each  trial  it  is  assumed  that  the  probability  is  a  constant  that 
the  system  will  fail  if  the  failure  mode  has  not  been  previously 
eliminated.  If  the  system  does  not  fail,  no  corrected  action  is  per¬ 
formed  before  the  next  trial.  If  the  system  fails,  then  an  attempt  is 
made  to  remove  the  failure  mode  from  the  system.  The  probability  of 
successfully  removing  the  failure  mode  is  also  assumod  to  be  a  constant 
for  each  attempt.  They  show  that  tho  system  reliability,  R  ,  on  the 
n-th  trial  is 


R 


n 


1-Ae 


-C(n-l) 


where  A  and  C  aro  parameters. 


Model  2.  Another  reliability  growth  model  was  considered  bv  Lloyd  and 
Lipow  (see  Appendix  B,  Reference  28)  where  the  development  program  is 
conducted  in  K  stages  and  on  the  i-th  stage  a  certain  number  of  systems 
are  tested.  The  reliability  growth  function  considered  was 


R.  =  R  -a/i, 

where  R.  is  the  system  reliability  during  the  i-th  stage,  Rw  is  the 
ultimate  reliability  as  i->»  and  a>0  is  a  parameter.  Maximum  likelihood 
and  least  squares  estimates  of  Rw  and  a  are  given  by  Lloyd  and  Lipow 
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along  with  a  lower  confidence  limit  for  R^,. 

Model  3.  Weiss  (see  Appendix  B,  Reference  41)  considered  a  reliability 
growth  model  where  the  mean  time  to  failure  of  a  system  with  exponential 
life  distribution  is  increased  by  removing  the  observed  failure  modes. 

In  particular,  he  showed  that  when  certain  conditions  hold,  the  increase 
of  the  mean  time  to  failure  is  approximately  at  a  constant  percent  per 
trial.  That  is,  if  0(i)  is  the  mean  time  to  failure  of  the  system  at 
trial  i  then  0(i)  may  be  approximated  under  certain  conditions  by 

0  (i)  =  AeCl, 

where  A  and  C  are  parameters.  Note  that 

0(i+l)  =  e^0(i). 

The  maximum  likelihood  estimates  of  A  and  C  are  given  by  Weiss. 

Model  4.  Wolman  (see  Appendix  B,  Reference  42)  considered  a  situation 
where  the  system  failures  are  classified  according  to  two  types.  The 
first  type  is  termed  "inherent  cause"  and  the  second  type  is  termed 
"assignable  cause".  Inherent  cause  failures  reflect  the  state-of-the 
art  and  may  occur  on  any  trial  while  assignable  cause  failures  may  be 
eliminated  by  corrective  action,  never  to  appear  again.  Wolman  assumed 
that  the  number  of  original  assignable  cause  failures  is  known  and  that 
whenever  one  of  these  modes  contribute  a  failure,  the  mode  is  removed 
permanently  from  the  system.  Wolman  uses  a  Markov-chain  approach  to 
derive  the  reliability  of  the  system  at  the  n-th  trial  when  the  failure 
probabilities  are  known. 

Model  5.  Barlow  and  Scheuer  (see  Appendix  B,  Reference  3)  considered 
a  nonparametric  model  for  estimating  the  reliability  of  a  system  during 
a  development  program.  They  assumed  that  the  design  and  engineering 
changes  do  not  decrease  the  system's  reliability,  hut,  unlike  some 
other  models,  they  do  not  fit  a  prescribed  functional  form  to  the 
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reliability  growth.  Their  model  is  similar  to  Wolman's  in  that  each 
failure  must  be  classified  either  as  inherent  or  assignable  cause. 

It  is  further  assumed  that  the  development  program  is  conducted  in 
K  stages,  with  similar  systems  being  tested  within  each  stage.  For 
each  stage,  the  number  of  inherent  failures,  the  number  of  assignable 
cause  failures  and  the  number  of  successes  are  recorded.  In  addition, 
they  assumed  that  the  probability  of  an  inherent  failure,  qQ,  remains 
the  same  throughout  the  development  program  and  that  the  probability  of 
an  assignable  cause  failure,  q^,  in  the  i-th  stage  does  not  increase 
from  stage  to  'stage  of  the  development  program.  The  authors  obtained 
the  maximum  likelihood  estimates  of  qQ  and  of  the  qj's  subject  to  the 
condition  that  they  be  nonincreasing.  A  conservative  lower  confidence 
bound  for  the  reliability  of  the  system  in  its  final  configuration  was, 
also,  given. 

Model  6.  Virene  (see  Appendix  B,  Reference  38)  considered  the  suitability 
of  the  Gompertz  equation 

t 

R  =  abc  , 

0<b<l,0<c<l,  for  reliability  growth  modeling.  In  this  equation 
a  is  the  upper  limit  approached  by  the  reliability  R  as  the  development 
time  t-*».  The  parameters  a,  b  and  c  are  unknown.  Virene  gave  estimates 
of  these  parameters  and  demonstrated  by  examples  the  application  of  this 
model. 

Model  7,  Duane  (see  Appendix  B,  Reference  17) considered  a  deterministic 
approach  to  reliability  growth  modeling.  He  analyzed  data  available 
for  several  systems  developed  by  General  Electric  in  an  effort  to 
determine  if  any  systematic  changes  in  reliability  improvement  occurred 
during  the  development  programs  for  these  systems.  His  analysis  revealed 
that  for  these  systems,  the  cumulative  failure  rate  versus  cumulative 
operating  hours  fell  close  to  a  straight  line  when  plotted  on  log-log 
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paper.  The  cumulative  failure  rate  appeared  to  decrease  at  approximately 
the  -0.4  or  -0.5  power  of  cumulative  operating  hours. 

The  types  of  systems  investigated  were  of  the  complex  electro¬ 
mechanical  and  mechanical  nature.  Duane  concluded  that  a  line  with  a 
slope  of  -0.5  representing  cumulative  failure  rate  as  a  function  of 
cumulative  operating  hours  on  log-log  paper  would  probably  be  suitable 
for  reflecting  reliability  growth  for  similar  type  systems  developed  at 
General  Electric. 

Mathematically,  Duane's  failure  rate  equation  may  be  expressed  by 

A(T)  =  KT_ct, 

K  >  0,  0  <_£*<_  1,  where  A(T)  is  the  cumulative  failure  rate  of  the 
system  at  operating  time  T,  and  K  and  a  are  parameters.  It 
follows  then  that 


X(T) 


E(T) 

T 


where  E(T)  is  the  expected  number  of  failures  the  system  will 
experience  during  T  units  of  operation.  This  yields 

E(T)  =  KT1_a. 


Furthermore,  the  instantaneous  failure  rate  at  T  is  given  by 

0(T)  =  Cl-a)KT"a. 


For  a  system  with  a  constant  failure  rate  the  mean  time  between  failure 
CMTBF)  of  the  system  at  operating  time  T  is 

MCT)  =  [OCT)]"1  -  [Cl-a)K]_1Ta, 


That  is,  the  change  in  system  MTBF  during  development  is  proportional 
to  Ta. 
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With  this  notation  o  ■  0.5  closely  represented  the  types  of  systems 
considered  by  Duane. 

Model  8.  Pollock  (see  Appendix  B,  Reference  32)  considered  a  Bayesian 
reliability  growth  model  for  a  system  undergoing  development.  The 
parameters  of  the  model  are  assumed  to  be  random  variables  with 
appropriate  prior  distribution  functions.  Using  his  results,  one  may 
project  the  system  reliability  to  any  time  after  the  start  of  the 
development  program  without  data  and,  also,  estimate  the  system 
reliability  after  data  have  been  observed.  He  further  gave  precision 
statements  regarding  the  projection  and  estimation. 


Model  9.  Barlow,  Proschan  and  Scheuer  (see  Appendix  B,  Reference  2) 
considered  a  reliability  growth  model  which  assumes  that  a  system  is 
being  modified  at  successive  stages  of  development.  At  stage  i  the 
system  reliability  (probability  of  success)  is  p^  The  model  of 
reliability  growth  under  which  one  obtains  the  maximum  likelihood 


estimates  of  p^,p2 


PK  assumes  that 


Pi  iP2i 


That  is,  it  is  required  that  the  system  reliability  be  not  degraded 
from  stage  to  stage  of  development.  No  particular  mathematical  form 
of  growth  is  imposed  on  the  reliability.  In  order  to  obtain  a  con¬ 
servative  lower  confidence  bound  on  p^,  it  suffices  to  require  only 
that 


PK  i  max  p. . 
i<K 

That  is,  it  is  only  necessary  that  the  reliability  in  the  latest  stage 
of  development  be  at  least  as  high  as  that  achieved  earlier  in  the 
development  program. 

Data  consist  of  x^  successes  in  n^  trials  in  stage  i,  isl,...,K. 


A  variation  of  this  model  is  treated  in  Barlow  and  Scheuer  (see 
Model  S) .  In  that  model  two  types  of  failure,  inherent  and  assignable 
cause,  are  distinguished. 

Model  10.  Another  reliability  growth  model  considered  by  Barlow, 
Proschan  and  Scheuer  (see  Appendix  B,  Reference  2)  assumed  that  at 
stage  i  of  development  the  distribution  of  system  life  length  is  b\. 
The  model  of  reliability  growth  under  which  the  maximum  likelihood 
estimates  of  F^(t) ,F^ (t) , . . . , ( t )  are  obtained,  writing 

Fi(t)  =  l-F.(t) 
is 

FjCt)  1  P2(t)  1  ...  <  FK(t) 

for  a  fixed  t  ^  0.  In  order  to  obtain  a  conservative  upper  confidence 
curve  on  FR(t)  and  thereby,  a  conservative  lower  confidence  curve  on 
FK(t)  for  all  non-negative  values  on  t,  it  suffices  only  to  require 
that 


PK(t)  >_  max  F.  (t) 
i<K  1 


for  all  t  ^  0.  That  is,  the  probability  of  system  survival  beyond  any 
time  t  in  the  latest  stage  of  development  is  at  least  as  high  as  that 
achieved  earlier  in  the  development  program. 

Data  consist  of  independent  life  length  observations 

Xil»...,Xin  ,i=l,...,K. 
i 

Model  11.  Bariow,  Proschan  and  Scheuer  (see  Appendix  B,  Reference  2), 
also,  considered  a  reliability  growth  model  which  assumes  that  the 
system  life  at  the  i-th  stage  of  development  has  increasing  failure  rate. 
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Because  of  improvement  from  stage  to  stage 


rl(t)  i.r2(t)  >_  ...  ir^t) 

for  t  >  0,  where  r^t)  is  the  failure  rate  at  time  t  at  the  i-th  stage 
of  development.  That  is,  for  each  t  >_ 0,  the  probability  of  system 
failure  in  the  interval  (t,  t+dt),  given  survival  till  time  t,  does  not 
increase  from  stage  to  stage  of  the  development  program. 

Given  life- length  observations,  X^,X^ X^n>  the  maximum 
likelihood  estimates  of  r1(t) ,r2(t) ,rK(t)  are  obtained. 
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